Video search system and method

ABSTRACT

Provided is a video search system and method. The video search system includes: a search engine configured to extract video data matching a search condition from an original video; and a browsing engine configured to generate a summarized video based on the extracted video data, render a plurality of objects in the summarized video on a background model in an overlapping manner in an object appearing order in the original video, and display the rendered objects.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from Korean Patent Application No.10-2013-0134994, filed on Nov. 7, 2013, in the Korean IntellectualProperty Office, the disclosure of which is incorporated herein in itsentirety by reference.

BACKGROUND

1. Field

Apparatuses and methods consistent with exemplary embodiments relate toa video search system and method.

2. Description of the Related Art

A video summary is made by configuring a short video via summarization.That is, the video summary is made by reducing a play length of anoriginal video. If a search result is composed of several videosections, a user may be able to determine details of the video sectionsafter reproducing the video of these sections. Such separatereproduction may cause inconvenience to the user, and may also betime-consuming. On the other hand, the video summary is a single videosummarized from the search result for reproduction, thereby providing auser with a convenient function of recognizing the details of the video.

SUMMARY

One or more embodiments of the inventive concept provide a search systemand method which allows user's convenient video search.

One or more exemplary embodiments will be set forth in part in thedescription which follows and the inventive concept will be apparentfrom the description, or may be learned by practice of the presentedembodiments.

According to an aspect of an exemplary embodiment, there is provided avideo search system which may include: a search engine configured toextract video data matching a search condition from an original video;and a browsing engine configured to generate a summarized video based onthe extracted video data, render a plurality of objects in thesummarized video on a background model in an overlapping manner in anobject appearing order in the original video, and display the renderedobjects.

The browsing engine may render the objects by changing a degree oftemporal overlapping between the objects in the summarized video whilemaintaining the object appearing order.

The browsing engine may limit the number of the objects displayed for apredetermined period of time.

The browsing engine may three-dimensionally render the summarized videointo a plurality of layers on a time or category basis in response to auser request.

The browsing engine may display the plurality of layers in anoverlapping manner and reproduce at least one layer selected from amongthe plurality of layers.

The browsing engine may remove at least one object corresponding to acanceling condition from the summarized video, if the cancelingcondition is input.

The browsing engine may display at least one object identified in thesummarized video as at least one search condition and remove an objectcorresponding to a search condition selected from the at least onesearch condition.

The video search system may further include an indexing engine forgenerating index data by structuralizing metadata extracted from theoriginal video, and the search engine may extract the index datamatching the search condition and the video data corresponding to theextracted index data.

According to an aspect of another exemplary embodiment, there isprovided a video search system which may include: a summarized videogeneration unit configured to generate a summarized video based on asearch condition and video data corresponding to the search conditionand extracted from original video, render a plurality of objects in thesummarized video on a background model in an overlapping manner in anobject appearing order in the original video, and display the renderedobjects; a three-dimensional (3D) configuring unit configured tothree-dimensionally arrange the summarized video into a plurality oflayers based on a time or category basis in response to a video changerequest; and a filtering unit configured to display the summarized videofrom which an object corresponding to a canceling condition is removed,if the canceling condition is input.

The summarized video generation unit may render the objects by changinga degree of temporal overlapping between the objects in the summarizedvideo while maintaining the object appearing order.

The summarized video generation unit may limit the number of the objectsdisplayed for a predetermined period of time.

The 3D configuring unit may the plurality of layers in an overlappingmanner and reproduce at least one layer selected from among theplurality of layers.

The filtering unit may display at least one object identified in thesummarized video as at least one search condition and remove an objectcorresponding to a search condition selected from the at least onesearch condition.

According to an aspect of still another exemplary embodiment, there isprovided a video search method which may include: extracting video datamatching a search condition from an original video; and generating asummarized video based on the extracted video data, rendering aplurality of objects in the summarized video on a background model in anoverlapping manner in an object appearing order in the original video,and displaying the rendered objects.

The rendering the object may include rendering the objects by changing adegree of temporal overlapping between the objects in the summarizedvideo while maintaining the object appearing order.

The rendering the object may include limiting the number of the objectsdisplayed for a predetermined period of time.

The video search method may further include three-dimensionallyrendering the summarized video into a plurality of layers on a time orcategory basis in response to a user request.

The three-dimensional rendering the summarized video may includedisplaying the plurality of layers in an overlapping manner andreproducing at least one layer selected from among the plurality oflayers.

The video search method may further include removing an objectcorresponding to a canceling condition from the summarized video, if thecanceling condition is input.

The video search method may further include displaying at least oneobject identified in the summarized video as at least one searchcondition and removing an object corresponding to a search conditionselected from the at least one search condition.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects will become apparent and more readilyappreciated from the following description of the exemplary embodiments,taken in conjunction with the accompanying drawings, in which:

FIG. 1 is a block diagram illustrating a video search system accordingto an exemplary embodiment;

FIG. 2 is a block diagram illustrating a structure of a browsing engineaccording to an exemplary embodiment;

FIG. 3 illustrates an example of displaying summarized video accordingto an exemplary embodiment;

FIG. 4 illustrates an example of a user input for switchover tosummarized video according to an exemplary embodiment;

FIG. 5 illustrates an example of a plurality of summarized video layersaccording to an exemplary embodiment;

FIGS. 6 and 7 are diagrams illustrating editing of summarized videoaccording to an exemplary embodiment;

FIG. 8 is a flowchart illustrating a video analysis method of a videosearch system according to an exemplary embodiment; and

FIGS. 9 and 10 are flowcharts illustrating a video search method of avideo search system according to an exemplary embodiment.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

Reference will now be made in detail to the exemplary embodiments, whichare illustrated in the accompanying drawings, wherein like referencenumerals refer to like elements throughout. In this regard, the presentembodiments may have different forms and should not be construed asbeing limited to the descriptions set forth herein. Accordingly, theembodiments are merely described below, by referring to the figures, toexplain aspects of the inventive concept. As used herein, the term“and/or” includes any and all combinations of one or more of theassociated listed items. Expressions such as “at least one of,” whenpreceding a list of elements, modify the entire list of elements and donot modify the individual elements of the list.

Hereinafter, various exemplary embodiments will be described withreference to the accompanying drawings. In the following description, adetailed description of known functions and configurations incorporatedherein will be omitted when it may obscure the subject matter of theembodiments.

Although ordinal numbers such as “first,” “second,” and so forth will beused to describe various components, those components are not limited bythe terms. The ordinal terms are used only for distinguishing onecomponent from another component.

The terminology used herein is for the purpose of describing embodimentsonly and is not intended to be limiting. As used herein, the singularforms are intended to include plural forms as well, unless the contextclearly indicates otherwise. It will be further understood that theterms “comprises” and/or “has” when used herein, specify the presence ofa stated feature, number, step, operation, component, element, or acombination thereof but do not preclude the presence or addition ofadditional features, numbers, steps, operations, components, elements,or combinations thereof.

The present embodiments may be described in terms of functional blockcomponents and various processing steps. Such functional blocks may berealized by any number of hardware and/or software components configuredto perform the specified functions. For example, the present embodimentsmay employ various integrated circuit components, e.g., memory elements,processing elements, logic elements, look-up tables, and the like, whichmay carry out a variety of functions under the control of one or moremicroprocessors or other control devices. Similarly, where the elementsof the present embodiments are implemented using software programming orsoftware elements the invention may be implemented with any programmingor scripting language such as C, C++, Java, assembler, or the like, withthe various algorithms being implemented with any combination of datastructures, objects, processes, routines or other programming elements.Functional aspects may be implemented with an algorithm executed in oneor more processors. Furthermore, the present invention could employconventional techniques for electronics configuration, signal processingand/or control, data processing, and the like. Terms such as“mechanism”, “element”, “means”, “component”, etc., may be used in abroad sense, and are not limited to mechanical and physical components.The terms may include a meaning of a series of routines of software inconnection with a processor or the like.

FIG. 1 is a block diagram illustrating a video search system 1 accordingto an exemplary embodiment. Referring to FIG. 1, the video search system1 includes an analysis unit 100, a database unit 300, and a search unit500.

The analysis unit 100 analyzes an input original video based onontology, and stores a result of the analysis in a form of metadata. Theanalysis unit 100 includes a video analysis engine 101 and an indexingengine 105.

The video analysis engine 101 analyzes the original video, classifiesthe original video according to a predefined condition such as apredefined category, and extracts attributes of an object detected fromthe original video, for example, a type, a color, a size, a form, amotion, and a trajectory of the object. A category may be a searchcondition that a user inputs for a desired such result. That is,according to an exemplary embodiment, a category and a search conditiondefined in the video search system 1 cooperate with each other, and if acategory is added, the search condition is also extended. The videosearch system 1 defines meanings of categories and a relationshipbetween the categories to accurately define a category of details to beanalyzed.

According the present embodiment, a summarized video is generatedthrough cooperation between a search condition and a category, therebyproviding, to the user, a semantic search condition that is the same asa search condition provided by the video search system 1. The user mayeasily recognize details from the summarized video.

The video analysis engine 101 performs a video analysis, such as sensingan appearance and disappearance of an object or sensing a motion of anobject, or video interruption in the original video. The video analysisengine 101 performs background region detection, foreground and objectdetection, object counting, camera tampering detection, and facedetection. The video analysis engine 101 calculates brightness, color,texture, and shape information about the video.

Moreover, the video analysis engine 101 generates an event if a resultof the video analysis satisfies an event generation condition. Herein,the event may include a system event that is set in a system, such aswhen a network error occurs or a new camera (not shown) is set. Theevent may also include a user event set by the user, such as appearingof an object, generation of an image specified by the user (for example,generation of a face that cannot be recognized), change of a screencolor, or occurrence of a motion in a preset region of the input video,and generation of an abnormal sound source (for example, a vehicle tirefrictional sound (skid), a glass-breaking sound, an alarm sound, animpact sound, and the like), generation of a user-specified sound source(for example, a shout, a scream, a cry, and the like), or generation ofa voice having a pitch greater than a threshold value for a soundsource.

The video analysis result is stored in the form of metadata in ametadata database 303. In addition to text-based metadata, image-basedmetadata such as a blob image in a motion region and a background modelmay also be stored in the metadata database 303.

To facilitate a search, the indexing engine 105 structuralizes andarranges metadata and stores the metadata in an index database 305.

The search unit 500 generates and provides a summarized video includinginformation that matches a search condition input from the user. Thesearch unit 500 may include a query engine 501, a search engine 503, anda browsing engine 505.

The query engine 501 receives the search condition from the user throughan input device 600, and analyzes user's requirements from the searchcondition and reconfigures the user's requirements in a predefined form.For example, the user may input a time period, a camera channel number,a type of an object (a person, a vehicle, or the like), a color, aparticular region (a region of interest (ROI)) on a screen, a particularline on the screen, and a particular direction, through the input device600.

The search engine 503 extracts index data that matches the reconfiguredsearch condition from the index database 305, and extracts video datacorresponding to the extracted index data from a video database 301.

The browsing engine 505 generates and renders in real time a summarizedvideo based on the extracted index data and video data. The browsingengine 505 displays time information (an object appearing time, an eventoccurring time, or the like) in the summarized video. The browsingengine 505 renders a background model and renders one or more movingobjects through overlapping on the background model.

FIG. 2 is a block diagram illustrating the browsing engine 505 accordingto an exemplary embodiment.

Referring to FIG. 2, the browsing engine 505 may include a summarizedvideo generation unit 510 and a summarized video editing unit 530.

The summarized video generation unit 510 generates and renders in realtime index summarized video based on the index data and the video datathat match the search condition.

Video summarization based on a time period, which is one of videosummarization methods, is a way to remove a time period during which nomoving object exists and to reconfigure the video. According toobject-based video summarization, an object in the original video mayappear in a different time period in the summarized video. That is, anobject appearing period is calculated and reconfigured for thesummarized video. According to time-period-based video summarization, atime period of the summarized video cannot be reduced much, such that ifthere is a motion in the whole time periods of the original video, thesummarized video has the same structure as the original video. Theobject-based video summarization may make it difficult to recognize anaction taken in the original video because the appearing time is in aquite different time period. For example, a moving line of one objectmay be divided into several separate moving lines that appear in thesummarized video. In addition, it is necessary to continuously calculatean appearing time of an appearing object in the summarized video, suchthat when a video of a long time is summarized, a number of objects needto be kept in a memory and the appearing order of the objects in thesummarized video needs to be continuously calculated.

According to the present embodiment, the summarized video generationunit 510 renders one or more objects extracted from the summarized videoaccording to the search condition based on the appearing order of theobjects in the original video. That is, the appearing order of theobjects in the summarized video maintains the appearing order of theobjects in the original video. For example, if an object B appears afteran object A in the original video, the browsing engine 505 may set theobject B to appear simultaneously with or after the object A in thesummarized video, and set the object B to at least avoid appearingbefore the object A. According to the present embodiment, the objectappearing order is maintained, such that the memory may be used on atime basis without keeping information about all objects (index data anddata of corresponding video) in the memory at the same time. That is,according to the present embodiment a limited queue is configured tokeep information about sequentially appearing objects in the memory,thereby efficiently using the memory, improving the speed of generatingthe summarized video, and linearly configuring the summarized video.

FIGS. 3 and 4 illustrate an example of displaying a summarized videoaccording to an exemplary embodiment.

Referring to FIG. 3, in an original video (a), an object A appears andthen disappears for a time T1, and after an elapse of a predeterminedperiod of time, an object B appears.

In summarized video I (b) and summarized video II (c), video data inwhich the object A and the object B appear is extracted to reduce theplay time of the original video into a time T2 and a time T3,respectively. The summarized video I (b) and the summarized video II (c)have different overlapping degrees between the object A and the objectB. Under a condition that the object B having a temporally laterappearing order than the object A does not appear before the object A,the degree of overlapping between the object A and the object B in thesummarized video may be adjusted by adjusting the complexity of thesummarized video.

The summarized video generation unit 510 renders the summarized video bylimiting the number of appearing objects. It may be confusing if toomany objects initially appear at the same time. Thus, the summarizedvideo generation unit 510 may configure the summarized video by limitingthe number of objects to, for example, two (2) at most, for severalseconds from the start of the summarized video, unless a plurality ofobjects appear at the same time in the original video.

The summarized video editing unit 530 changes the summarized video froma two-dimensional (2D) structure into a three-dimensional (3D) structureor from the 3D structure to the 2D structure or changes settings of thesummarized video. The summarized video editing unit 530 includes a 3Dconfiguring unit 550 and a filtering unit 570.

Upon receiving input of a video change request, the 3D configuring unit550 changes the 2D summarized video that is being reproduced into 3Dsummarized video configured of a plurality of summarized video layers ona time or category basis in response to the video change request. Thevideo change request may be a user input that draws a curve from theleft to the right or from the right to the left on a screen 700 a (FIG.4) on which the summarized video is being reproduced, by using a touchor the input device 600 such as a mouse.

FIG. 5 illustrates an example of rendering a plurality of summarizedvideo layers according to an exemplary embodiment.

Referring to FIG. 5, a summarized video having information about eventscorresponding to appearing of a vehicle in a particular region may bechanged into a plurality of summarized video layers corresponding to atype or color of the vehicle in response to the video change request.

For example, a single summarized video that is being reproduced, inwhich a plurality of vehicles appearing for a predetermined period oftime is rendered on a background model, may be changed into a pluralityof summarized video layers including a first layer (a) that is asummarized video of an event corresponding to appearance of a firstvehicle at 1:37 AM, a second layer (b) that is a summarized video of anevent corresponding to appearance of a second vehicle at 6:08 AM, and athird layer (c) that is a summarized video of an event corresponding toappearance of a third vehicle at 1:24 PM, in response to the imagechange request. The lowest layer of the screen is a background model BGand a summarized video indicating a temporally preceding event issituated at a higher layer. If the user views the screen, four layersare reproduced in an overlapping manner.

The 3D configuring unit 550 reproduces a summarized video of a layerselected by the user from among the plurality of layers. For example, ifthe user selects the third layer (c) from among the plurality of layersillustrated in FIG. 5, the summarized video of the third layer (c) maybe reproduced on the background model BG on the screen. The user mayselect one or more layers.

The 3D configuring unit 550 combines a plurality of 3D summarized videolayers into a single 2D summarized video and reproduces the 2Dsummarized video in response to the image change request. The imagechange request may be a drawing action taken in the same direction as orin a different direction than that of a drawing action for change into a3D structure, or may be an input that is set differently.

When a summarized video that is a result matching a user's query (searchcondition) is rendered in real time and reproduced, if a requestcanceling a particular search condition is input, a new summarized videomay be rendered in which details corresponding to the cancelingcondition are filtered (removed) from the reproduced summarized video.For example, if an initial search condition is white and red vehicles, asummarized video displaying both a white vehicle and a red vehicle isreproduced on the screen. If an input for canceling a white vehiclecondition is received, the filtering unit 570 removes the white vehiclefrom the summarized video that is being reproduced. Thus, on the screen,a summarized video displaying only the red vehicle is reproduced.

FIGS. 6 and 7 illustrate editing of a summarized video according to anexemplary embodiment.

Referring to FIG. 6, as illustrated in a left portion, a timeline forchecking a position (a play time) of a summarized video that is beingreproduced is provided and a high-layer category menu including videoanalytics (VA) selection, object, color, ROI, and the like is providedas a search condition. The user may move a play position by selecting ortouching a bar of the timeline or a time scale indicated on thetimeline.

Once one of the provided high-layer category menu is selected, a subcategory menu of the selected high-layer category is provided on aseparate right window as illustrated in a right portion or on thesummarized video that is being reproduced. For example, the colorcategory, which is a high-layer category, may include sub categoriessuch as white, black, red, yellow, green, and purple. Once a category isselected by user's selection, a summarized video including informationmatching the selected category may be generated and reproduced.

The user may make a selection for canceling at least one of colorspreviously selected during reproduction of the summarized video, and inresponse to the selection, the filtering unit 570 may generate asummarized video from which data corresponding to a color for whichcancelation is requested is removed. For example, if a selection forcanceling a sub category “yellow” is input from the user duringreproduction of a summarized video including information about vehiclesin all colors, the summarized video is updated with a summarized videofrom which a yellow vehicle is removed, and the updated summarized videois reproduced on the screen.

Referring to FIG. 7, if the object category is selected from thehigh-layer category menu illustrated in a left portion, a sub categorymenu of the selected object category is provided on a separate rightwindow or on the summarized video that is being reproduced. For example,the object category, which is the high-layer category, may include subcategories such a type category, a person category, a car category, andan anything category.

The filtering unit 570 provides a ‘Detected Object’ category as a subcategory to provide objects identified in generation of the summarizedvideo as a search condition. If an object the user does not desire tosearch for is identified in the summarized video, the user may makeselection for canceling at least one of the identified objects and inresponse, the filtering unit 570 generates a summarized video from whichdata corresponding to the canceling-requested object is removed.

The database unit 300 stores an original video and a video analysisresult. The database unit 300 may include a video database 301, ametadata database 303, and an index database 305.

The original video may be a video that is directly transmitted from acamera (not illustrated) which is a video sensor connected to the videosearch system 1, may be video stored in a digital video recorder (DVR)or a network video recorder (NVR) connected to the video search system1, or may be video input through various paths over a network (notillustrated).

The original video in the form of a data stream input to the videosearch system 1 is stored in the video database 301.

The analysis result of the original video is stored in the form oftext-based metadata in the metadata database 303. The metadata database303 stores image-based metadata that is a part of the original video,for example, a blob image of a motion-detected object and a backgroundmodel.

A user terminal 800 is a terminal used by a user or a security officialwho manages the video search system 1, and may be a personal computer(PC) or a mobile terminal. The user may control the video search system1 through the user terminal 800. The user terminal 800 includes theinput device 600 that is a user interface capable of inputting a query(search condition) to the video search system 1.

The input device 600 is wiredly or wirelessly connected with the videosearch system 1 such that the user generates input data for controllingan operation of the video search system 1. The input device 600 may be akeypad, a dome switch, a touch pad (of a contact-type capacitive type, apressure-type resistive type, an infrared sensing type, a surfaceultrasonic conduction type, an integral tension measurement type, apiezoelectric effect type, or the like), a mouse, a remote controller, ajog wheel, or a jog switch. The user terminal 800 sets a displaycondition of the summarized video by modifying, adding or deleting thesearch condition with the input device 600.

The user terminal 800 displays the summarized video matching the searchcondition on a display device 700. The display device 700 provides thesummarized video output from the browsing engine 505 to the user,allowing the user to monitor the displayed video. The display device 700displays a menu item for a user's direct or indirect selection of asearch condition such as change between a 2D structure and a 3Dstructure of the summarized video, a type and position of the object tobe displayed on the summarized video, or a particular time. The searchcondition may be extended by cooperating with a category set in thevideo analysis engine 101.

The display device 700 provides visual information and/or audibleinformation to the user. The display device 700 may include a liquidcrystal panel (LCD), an organic light emitting display (OLED) panel, oran electrophoretic display (EPD) panel. The display device 700 may beprovided in the form of a touchscreen to receive an input through auser's touch and may operate as an input interface.

FIG. 8 is a flowchart illustrating a video analysis method for the videosearch system 1 according to an exemplary embodiment.

Referring to FIG. 8, once an original video is input in operation S11,the video search system 1 analyzes the original video by using aclassification and analysis module in operation S13. The video searchsystem 1 may analyze the original video according to a predefinedcategory. The category may cooperate with a search condition and thesearch condition may be set and extended according to a definedcategory.

The video search system 1 generates and stores metadata that is a resultof the analysis of the original video in operation S15. The metadata mayinclude image-based metadata such as a block image and a backgroundmodel, as well as text-based metadata.

The video search system 1 generates and stores index data bystructuralizing and arranging the metadata to facilitate the search inoperation S17.

FIG. 9 is a flowchart illustrating a video search method for the videosearch system 1 according to an exemplary embodiment.

Referring to FIG. 9, once a search condition is input from the user inoperation S31, the video search system 1 generates and reproduces asummarized video that matches the search condition in operation S33. Thevideo search system 1 extracts index data that matches the searchcondition and extracts video data corresponding to the index data fromthe original video. The video search system 1 generates a summarizedvideo based on the extracted video data.

The video search system 1 renders the background model, and in real timerenders a moving object in an overlapping manner on the backgroundmodel, thereby reproducing the summarized video. The video search system1 may determine an object appearing order in the summarized video tocorrespond to an object appearing order in the original video, therebyrendering the object. The video search system 1 renders the summarizedvideo by limiting the number of appearing objects for several secondsfrom the start of the summarized video, unless a plurality of objectsappear at the same time in the original video.

Upon input of the video change request in operation S35, the videosearch system 1 changes a 2D summarized video into a 3D structure inwhich one or more summarized video layers overlap in operation S37. Thevideo search system 1 may change the 2D summarized video that is beingreproduced into a plurality of summarized video layers arranged on atime or category basis, in response to the video change request.

Upon receiving a selection of one of the plurality of layers, the videosearch system 1 reproduces a summarized video of the selected layer inoperation S39. The user may select one or more layers at the same time.The one or more selected layers may be combined and reproduced as asingle summarized video.

In a structure where the plurality of summarized video layers overlap,upon input of a request for restoring the 2D summarized video, the videosearch system 1 restores and reproduces the original 2D summarized videoincluding information about the plurality of layers.

FIG. 10 is a flowchart illustrating a video search method for the videosearch system 1 according to an exemplary embodiment.

Referring to FIG. 10, upon input of a search condition from the user inoperation S51, the video search system 1 generates and reproduces asummarized video that matches a search condition in operation S53. Thevideo search system 1 extracts index data that matches the searchcondition and extracts video data corresponding to index data from theoriginal video. The video search system 1 generates the summarized videobased on the extracted video data.

The video search system 1 renders a background model and in real timerenders a moving object in an overlapping manner on the backgroundmodel, thereby reproducing a summarized video. The video search system 1may determine an object appearing order in the summarized video tocorrespond to an object appearing order in the original video, therebyrendering the object. The video search system 1 renders the summarizedvideo by limiting the number of appearing objects for several secondsfrom the start of the summarized video, unless a plurality of objectsappear at the same time in the original video.

If a change to the search condition is input in operation S55, the videosearch system 1 updates the summarized video according to the changedsearch condition and reproduces an updated summarized video in operationS57. If a request for canceling the search condition is input duringreproduction of the summarized video that matches the user's searchcondition, the video search system 1 renders a new summarized video inwhich details corresponding to the canceling condition are filtered(removed) from the summarized video that is being reproduced. The videosearch system 1 displays one or more objects identified in thesummarized video as a search condition, and upon the input of acanceling request, the video search system 1 updates the summarizedvideo with the summarized video from which an object corresponding tothe canceling-requested search condition is removed, and reproduces theupdated summarized video.

The above embodiments may improve accuracy and easiness of searching bysimultaneously applying multiple filtering to the summarized video,easily changing a summarized video, and applying changes to thesummarized video in real time.

As described above, according to the one or more of the aboveembodiments, the user may conveniently input a search condition andeasily recognize details from a search result.

The computer readable code can be recorded/transferred on media in avariety of ways, with examples of the media including recording media,such as magnetic storage media (e.g., ROM, floppy disks, hard disks,etc.) and optical recording media (e.g., CD-ROMs, or DVDs), andtransmission media such as Internet transmission media. Thus, a mediummay be such a defined and measurable structure including or carrying asignal or information, such as a device carrying a bitstream accordingto one or more embodiments. The media may also be a distributed network,so that the computer readable code may be stored/transferred andexecuted in a distributed fashion. A functional program and code andcode segments may be easily construed by programmers of ordinary skillin the art.

It should be understood that the exemplary embodiments described hereinshould be considered in a descriptive sense only and not for purposes oflimitation. Descriptions of features or aspects within each embodimentshould typically be considered as available for other similar featuresor aspects in other embodiments.

While one or more exemplary embodiments have been described withreference to the figures, it will be understood by those of ordinaryskill in the art that various changes in form and details may be madetherein without departing from the spirit and scope of the inventiveconcept as defined by the following claims.

What is claimed is:
 1. A video search system comprising: a search engineconfigured to extract video data matching a search condition from anoriginal video; and a browsing engine configured to: generate asummarized video based on the extracted video data, render, in realtime, a plurality of objects over a background model in the summarizedvideo in an overlapping manner according to a temporal order ofappearance of the plurality of objects in the original video, displaythe plurality of rendered objects over the background model, thetemporal order of appearance of the plurality of objects in the originalvideo corresponding to a displaying order of the plurality of renderedobjects in the summarized video, and, in response to a video changerequest, display a plurality of summarized video layers in theoverlapping manner based on at least one of a time basis and a categorybasis, the background model corresponding to a lowest layer of theplurality of summarized video layers.
 2. The video search system ofclaim 1, wherein the browsing engine is configured to render theplurality of objects by changing a degree of temporal overlappingbetween the plurality of objects in the summarized video whilemaintaining the displaying order of the plurality of objects in thesummarized video.
 3. The video search system of claim 1, wherein thebrowsing engine is configured to limit a number of the plurality ofobjects displayed for a predetermined period of time.
 4. The videosearch system of claim 1, wherein the browsing engine is configured todisplay at least one selected layer of the plurality of summarized videolayers, in response to a selection from among the plurality ofsummarized video layers displayed in the overlapping manner over thebackground model.
 5. The video search system of claim 1, wherein thebrowsing engine is configured to remove at least one objectcorresponding to a canceling condition from the summarized video, if thecanceling condition is input.
 6. The video search system of claim 1,wherein the browsing engine is configured to display at least one objectidentified in the summarized video as at least one search condition andremove an object corresponding to a search condition selected from theat least one search condition.
 7. The video search system of claim 1,further comprising an indexing engine configured to generate index databy structuralizing metadata extracted from the original video, whereinthe search engine is further configured to extract the index datamatching the search condition and the video data corresponding to theextracted index data.
 8. A video search system comprising: a summarizedvideo generation unit configured to: generate a summarized video basedon a search condition and video data corresponding to the searchcondition, the video data being extracted from an original video,render, in real time, a plurality of objects in the summarized video inan overlapping manner according to a temporal order of appearance of theplurality of objects in the original video, and display the plurality ofrendered objects over the background model, the temporal order ofappearance of the plurality of objects in the original videocorresponding to a displaying order of the plurality of rendered objectsin the summarized video; a three-dimensional (3D) configuring unitconfigured to three-dimensionally arrange the summarized video into aplurality of summarized video layers in the overlapping manner over thebackground model based on one of a time basis and a category basis, inresponse to a video change request, the background model correspondingto a lowest layer of the plurality of summarized video layers; and afiltering unit configured to display the summarized video from which anobject corresponding to a canceling condition is removed, in response tothe canceling condition being input.
 9. The video search system of claim8, wherein the summarized video generation unit is configured to renderthe plurality of objects by changing a degree of temporal overlappingbetween the plurality of objects in the summarized video whilemaintaining the displaying order of the plurality of objects in thesummarized video.
 10. The video search system of claim 8, wherein thesummarized video generation unit is configured to limit a number of theplurality of objects displayed for a predetermined period of time. 11.The video search system of claim 8, wherein the 3D configuring unit isfurther configured to display at least one selected layer of theplurality of summarized video layers, in response to a selection fromamong the plurality of summarized video layers displayed in theoverlapping manner over the background model.
 12. The video searchsystem of claim 8, wherein the filtering unit is configured to displayat least one object identified in the summarized video as at least onesearch condition and remove an object corresponding to a searchcondition selected from the at least one search condition.
 13. A videosearch method comprising: extracting video data matching a searchcondition from an original video; generating a summarized video based onthe extracted video data; rendering, in real time, a plurality ofobjects over a background model in the summarized video in anoverlapping manner according to a temporal order of appearance of theplurality of objects in the original video; displaying the renderedplurality of objects over the background model, the temporal order ofappearance of the plurality of objects in the original videocorresponding to a displaying order of the rendered plurality of objectsin the summarized video; and in response to a video change request,displaying a plurality of summarized video layers in the overlappingmanner over the background model based on at least one of a time basisand a category basis, the background model corresponding to a lowestlayer of the plurality of summarized video layers.
 14. The video searchmethod of claim 13, wherein the rendering the plurality of objectscomprises rendering the plurality of objects by changing a degree oftemporal overlapping between the plurality of objects in the summarizedvideo while maintaining a displaying order of appearance of theplurality of objects in the summarized video.
 15. The video searchmethod of claim 13, wherein the rendering the plurality of objectscomprises limiting a number of the objects displayed for a predeterminedperiod of time.
 16. The video search method of claim 13, furthercomprising displaying at least one selected layer of the plurality ofsummarized video layers, in response to a selection from among theplurality of summarized video layers displayed in the overlapping mannerover the background model.
 17. The video search method of claim 13,further comprising removing an object corresponding to a cancelingcondition from the summarized video, if the canceling condition isinput.
 18. The video search method of claim 13, further comprisingdisplaying at least one object identified in the summarized video as atleast one search condition and removing an object corresponding to asearch condition selected from the at least one search condition.